Video event extraction aims to detect salient events from a video and identify the arguments for each event as well as their semantic roles. Existing methods focus on capturing the overall visual scene of each frame, ignoring fine-grained argument-level information. Inspired by the definition of events as changes of states, we propose a novel framework to detect video events by tracking the changes in the visual states of all involved arguments, which are expected to provide the most informative evidence for the extraction of video events. In order to capture the visual state changes of arguments, we decompose them into changes in pixels within objects, displacements of objects, and interactions among multiple arguments. We further propose Object State Embedding, Object Motion-aware Embedding and Argument Interaction Embedding to encode and track these changes respectively. Experiments on various video event extraction tasks demonstrate significant improvements compared to state-of-the-art models. In particular, on verb classification, we achieve 3.49% absolute gains (19.53% relative gains) in F1@5 on Video Situation Recognition.

图神经网络（GNN）在图形分类和多样化的下游现实世界应用方面取得了巨大成功。尽管他们成功了，但现有的方法要么仅限于结构攻击，要么仅限于本地信息。这要求在图形分类上建立更一般的攻击框架，由于使用全球图表级信息生成本地节点级的对抗示例的复杂性，因此面临重大挑战。为了解决这个“全局到本地”问题，我们提出了一个通用框架CAMA，以通过层次样式操纵图形结构和节点特征来生成对抗性示例。具体而言，我们利用Graph类激活映射及其变体来产​​生与图形分类任务相对应的节点级的重要性。然后，通过算法的启发式设计，我们可以借助节点级别和子图级的重要性在不明显的扰动预算下执行功能和结构攻击。在六个现实世界基准上攻击四个最先进的图形分类模型的实验验证了我们框架的灵活性和有效性。

节点分类和图形分类是两个图形学习问题，分别预测节点的类标签和图形的类标签。图的节点通常代表现实世界实体，例如，社交网络中的用户或文档引用网络中的文档。在这项工作中，我们考虑了一个更具挑战性但实际上有用的设置，其中节点本身是图形实例。这导致了层次图的观点，该视角在许多领域（例如社交网络，生物网络和文档收集）中产生。我们在层次图中研究节点分类问题，其中“节点”是图形实例。由于标签通常受到限制，我们设计了一种新型的半监督溶液，名为Seal-CI。 Seal-CI采用了一个迭代框架，该框架需要轮流更新两个模块，一个模块在图形实例级别，另一个在层次图级别上进行。为了在不同级别的层次图之间执行一致性，我们提出了分层图共同信息（HGMI），并进一步提出了一种使用理论保证计算HGMI的方法。我们证明了该层次图建模的有效性以及在文本和社交网络数据上提出的密封CI方法。

Vision-Language（V + L）预先润廓模型通过了解图像和文本之间的对齐来支持多媒体应用程序取得了巨大成功。虽然现有的视觉预押模型主要专注于了解文本中的图像或实体中的对象，但它们通常会忽略事件级别的对齐及其参数结构。 ％在这项工作中，我们提出了一种对比的学习框架来强制执行愿景 - 语言预押模型来理解事件和相关参数（参与者）角色。为此，我们利用文本信息提取技术来获得事件结构知识，并利用多个提示函数来通过操纵事件结构来对比难度的负面描述。我们还基于最佳传输来设计事件图对齐损耗以捕获事件参数结构。此外，我们收集了一个大型活动的数据集（106,875张图片），用于预磨平，这提供了更具挑战性的图像检索基准，以评估对复杂冗长的句子的理解。实验表明，我们的零射剪辑事件优于在多媒体事件提取中的参数提取中的最先进的监督模型，从而实现了事件提取中的5±绝对f得分增益，以及显着改进零拍摄设置下的各种下游任务。

最近，对建立问题的兴趣越来越兴趣，其中跨多种模式（如文本和图像）的原因。但是，使用图像的QA通常仅限于从预定义的选项集中挑选答案。此外，在现实世界中的图像，特别是在新闻中，具有与文本共同参考的对象，其中来自两个模态的互补信息。在本文中，我们提出了一种新的QA评估基准，并在新闻文章中提出了1,384个问题，这些文章需要跨媒体接地图像中的物体接地到文本上。具体地，该任务涉及需要推理图像标题对的多跳问题，以识别接地的视觉对象，然后从新闻正文文本中预测跨度以回答问题。此外，我们介绍了一种新颖的多媒体数据增强框架，基于跨媒体知识提取和合成问题答案生成，自动增强可以为此任务提供弱监管的数据。我们在我们的基准测试中评估了基于管道和基于端到端的预先预测的多媒体QA模型，并表明他们实现了有希望的性能，而在人类性能之后大幅滞后，因此留下了未来工作的大型空间，以便在这一具有挑战性的新任务上的工作。

The recent increase in public and academic interest in preserving biodiversity has led to the growth of the field of conservation technology. This field involves designing and constructing tools that utilize technology to aid in the conservation of wildlife. In this article, we will use case studies to demonstrate the importance of designing conservation tools with human-wildlife interaction in mind and provide a framework for creating successful tools. These case studies include a range of complexities, from simple cat collars to machine learning and game theory methodologies. Our goal is to introduce and inform current and future researchers in the field of conservation technology and provide references for educating the next generation of conservation technologists. Conservation technology not only has the potential to benefit biodiversity but also has broader impacts on fields such as sustainability and environmental protection. By using innovative technologies to address conservation challenges, we can find more effective and efficient solutions to protect and preserve our planet's resources.

We present Muse, a text-to-image Transformer model that achieves state-of-the-art image generation performance while being significantly more efficient than diffusion or autoregressive models. Muse is trained on a masked modeling task in discrete token space: given the text embedding extracted from a pre-trained large language model (LLM), Muse is trained to predict randomly masked image tokens. Compared to pixel-space diffusion models, such as Imagen and DALL-E 2, Muse is significantly more efficient due to the use of discrete tokens and requiring fewer sampling iterations; compared to autoregressive models, such as Parti, Muse is more efficient due to the use of parallel decoding. The use of a pre-trained LLM enables fine-grained language understanding, translating to high-fidelity image generation and the understanding of visual concepts such as objects, their spatial relationships, pose, cardinality etc. Our 900M parameter model achieves a new SOTA on CC3M, with an FID score of 6.06. The Muse 3B parameter model achieves an FID of 7.88 on zero-shot COCO evaluation, along with a CLIP score of 0.32. Muse also directly enables a number of image editing applications without the need to fine-tune or invert the model: inpainting, outpainting, and mask-free editing. More results are available at https://muse-model.github.io

Domain adaptive detection aims to improve the generalization of detectors on target domain. To reduce discrepancy in feature distributions between two domains, recent approaches achieve domain adaption through feature alignment in different granularities via adversarial learning. However, they neglect the relationship between multiple granularities and different features in alignment, degrading detection. Addressing this, we introduce a unified multi-granularity alignment (MGA)-based detection framework for domain-invariant feature learning. The key is to encode the dependencies across different granularities including pixel-, instance-, and category-levels simultaneously to align two domains. Specifically, based on pixel-level features, we first develop an omni-scale gated fusion (OSGF) module to aggregate discriminative representations of instances with scale-aware convolutions, leading to robust multi-scale detection. Besides, we introduce multi-granularity discriminators to identify where, either source or target domains, different granularities of samples come from. Note that, MGA not only leverages instance discriminability in different categories but also exploits category consistency between two domains for detection. Furthermore, we present an adaptive exponential moving average (AEMA) strategy that explores model assessments for model update to improve pseudo labels and alleviate local misalignment problem, boosting detection robustness. Extensive experiments on multiple domain adaption scenarios validate the superiority of MGA over other approaches on FCOS and Faster R-CNN detectors. Code will be released at https://github.com/tiankongzhang/MGA.

An unbiased scene graph generation (SGG) algorithm referred to as Skew Class-balanced Re-weighting (SCR) is proposed for considering the unbiased predicate prediction caused by the long-tailed distribution. The prior works focus mainly on alleviating the deteriorating performances of the minority predicate predictions, showing drastic dropping recall scores, i.e., losing the majority predicate performances. It has not yet correctly analyzed the trade-off between majority and minority predicate performances in the limited SGG datasets. In this paper, to alleviate the issue, the Skew Class-balanced Re-weighting (SCR) loss function is considered for the unbiased SGG models. Leveraged by the skewness of biased predicate predictions, the SCR estimates the target predicate weight coefficient and then re-weights more to the biased predicates for better trading-off between the majority predicates and the minority ones. Extensive experiments conducted on the standard Visual Genome dataset and Open Image V4 \& V6 show the performances and generality of the SCR with the traditional SGG models.

With the increasing ability of large language models (LLMs), in-context learning (ICL) has become a new paradigm for natural language processing (NLP), where LLMs make predictions only based on contexts augmented with a few training examples. It has been a new trend exploring ICL to evaluate and extrapolate the ability of LLMs. In this paper, we aim to survey and summarize the progress, challenges, and future work in ICL. We first present a formal definition of ICL and clarify its correlation to related studies. Then, we organize and discuss advanced techniques of ICL, including training strategies, prompting strategies, and so on. Finally, we present the challenges of ICL and provide potential directions for further research. We hope our work can encourage more research on uncovering how ICL works and improving ICL in future work.